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IBSTBICT 

Graduating senior ratings uere added to colleague and 
currently enrolled student ratings (gathered on a group of 417 
instructors in an earlier study) and then conpared to the 
instructors* research productivitj and acadesic rank. Graduating 
senior, colleague, and student ratings were not found to be 
significantly related to the instructors* research productivit]. 
Bovever, senior ratings were significantly and highly related to 
colleague and currently enrolled student ratings but net to acadenic 
rank. Indicating that the reputation of the instructors say not be 
influencing seniors* judgnents of excellence in teaching. (Author) 
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Abstract 

Graduating senior ratings were added to colleague and currently enrolled 
student ratings (gathered on a group of 477 Instructors In an earlier study) and 
then compared to the Instructors* research productivity and academic rank. 
Graduating senior, colleague and student ratings were not found to be significantly 
related to the Instructors* research productivity. However, senior ratings were 
significantly and highly related to colleague and currently enrolled student 
ratings but not to academic rank Indicating that the reputation of the Instructors 
may not be Influencing seniors* Judgments of excellence In teaching* 
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GRADUATING SENIOR RATINGS RELATIONSHIP TO COLLEAGUE RATING, 
STUDENT RATING, RESEARCH PRODUCTIVITY 
AND ACADEMIC RANK IN RATING INSTRUCTIONAL EFFECTIVENESS 
Lawrence M« Aleanonl and Makonnen Ylmer 

In a recent study by Aleaoonl and Ylmer (1973), teacher effectiveness was 
Investigated with respect to the relationships between current student ratings, 
ratings by fellow teachers (colleagues) , and research productivity # Their 
results suggested that teachers and students differ In the basis of their 
rating since teachers appeared to take Into account academic rank of the 
Instructor In their rating while the students did not. Research productivity 
was not related to either student or colleague rating. That study did not 
report ratings by graduating seniors and/or alumni, 

McReachle (1969) argued that students are more apt to accurately evaluate 
their teachers or the Instruction they received after they have left college and 
gotten some perspective on what was really valuable to them. However^ McKeachle 
was not able to supply evidence to support his contention, Drucker and ReoDera 
(1950; 1951), however, conducted a study that asked alumni, ten years out of 
college, what they thought the most important qualities of a good instructor 
were and then compared their answers with current undergraduates* opinions. 
They found that there was a positive correlation between current student and 
alumni ratings of the same Instructor Indicating that the student ratings after 
they have left college are still similar to the ratings of students presently 
in college, 

authors are Indebted to Professor Robert A. Waller for providing the 
Graduating Senior Rating data. 
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A more recent study by Gaff (1973) using graduating seniors' nominations 
of stimulating teachers from nine California Instltuclons found "...a fair 
degree of overlap between the faculty nominated by students and those named by 
colleagues." No correlations, however, were presented. 

Long after the Aleamonl and Ylmer study was completed, data from a senior 
questionnaire used on graduating seniors In the College of Liberal Arts and 
Sciences (LAS) at the University of Illinois at Urbana*-Champalgn was made 
available to the authors. The Senior Questionnaire contained questions asking 
students to nominate their "most stimulating" and "least stimulating" teachers. 
A review of the literature Indicated that no studies were available that 
concentrated on the concurrent relationship between graduating senior ratings , 
colleague ratings, current student ratings, research productivity and academic 
rank. Using the available data gathered from graduating seniors, this study 
was designed to examine these relationships. 

Method 

The method section of this study is Identical to that described in the 
earlier study by Aleamonl and Ylmer except that five additional variables are 
employed which were gathered from the LAS Senior Questionnaires administered to 
the 1969, 1970, 1971, and 1972 graduating seniors. The graduating seniors were 
asked to Indicate their "itt>st stimulating" and "least stimulating" teachers 
during their tenure at the University. Only those teachers who received 
"most stimulating" nominations were used since that was the same criterion 
used to select the faculty nominated group in the earlier study. Individual 
faculty members were rated according to the frequency of nomination which 
ranged from 1 to 55, 1 to 110, 1 to 54, and 1 to 52 for the 1969, 1970, 1971, 
and 1972 groups, respectively. The nuiid)er of graduating seniors filling out 

ERIC 



3. 

the questionnaire in 1969, 1970, 1971, and 1972 were 2,205, 2,245, 2,562, 
and 2,441, respectively. 

A correlational analysis again was used along with a multiple regression 
on the variables of interest. 

Results 

A description of the variables analyzed as well as their means (M) and 
standard deviations (SD) is presented in Table 1. The distributions of senior 
ratings was positively skewed and was similar to that of the colleague rating. 
The 1970 senior rating exhibited the largest shewness. In Table 2, the data 
above the main diagonal represent the intercorrelations among the variables, 
while the data below the main diagonal represent the corresponding san^le sizes • 

Since the Instructor was the focus of the ratings by both the graduating 
seniors and colleague^. Variables 10 (Advisor Instructor) and 15 (CEQ Instructor) 
will be used as measures of enrolled student ratings of teacher effectiveness 
In this study without considering the rest of the Advisor or CEQ variables. 

Table 3 presents the intercorrelations of Variables 1, 3, 4, 7, 8, 10, 15, 
and 18. The correlation of the Senior Rating Total (Variable 8) with Advisor 
Instructor and CEQ Instructor is .26 and .17, respectively. The 1969 Senior 
Rating correlates .17 and .23 with Advisor Instructor and CEQ Instructor, 
respectively, while the 1972 rating correlators .19 and .38, respectively. 
The correlation of Variable 3 (Colleague Rating) with Academic Rank, Advisor 
Instructor, CEQ Instructor and Senior Rating Total is .20, .28, .27, and .42, 
respectively, and all of them are significant at the p < .05 level. The 
student*colleague correlation of .28 is significantly different from the 
senior-colleague correlation of «42 at the p < .05 level. Advisor Instructor, 
CEQ Instructor, Publications and Senior Rating in predicting Colleague Rating, 
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Table 1 

Variable Description, Mean and Standard Deviation 



Variable 


N 


M 


SD 


1. 


Academic Rank 




477 


3.83 


1.17 


2. 


Sex 




477 


1.12 


.32 


3. 


Colleague Rating 




477 


2.98 


3.14 


Senior Rating 


4. 


1969 




192 


7.54 


9.46 


5. 


1970 




220 


11.01 


16.23 


6. 


1971 




193 


8.60 


10.66 


7. 


1972 




176 


7.12 


8.79 


8. 


Total 




256 


26.50 


37.22 


Advisor Subscales 


9. 


Overall Evaluation of Course 


474 


3.11 


.28 


10. 


Instructor 




474 


3.22 


.39 


CEQ Subscales 


11. 


General Course Attitude 


43 


3.16 


.35 


12. 


Method of Instruction 




43 


2.84 


.42 


13. 


Course Content 




43 


2.97 


.24 


14. 


Interest and Attention 




43 


2.85 


.42 


15. 


Instructor 




43 


3.22 


.34 


16. 


Specific Items 




43 


2.95 


.20 


17. 


Total 




43 


3.00 


.30 


Publications 


18. 


Unweighted Sum 




362 


7.89 


7.55 


19. 


Weighted Sum I 




362 


28.32 


28.99 


20. 


Weighted Sum II 




362 


25.27 


24.33 



o 
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Table 3 
Selected Intercorrelaclons 



Variable 


1 


A 


7 


8 


10 


15 


18 


3 


1. 


Academic Rank 


1.00 


-.06 


-.14 


-.06 


.03 


-.05 


.32** 


.20** 


4. 


Senior Rating 1969 




1.00 


.40** 


.83** 


.17* 


.23 


-.02 


.45** 


7. 


Senior Rating 1972 






1.00 


.75** 


.19** 


.38 


-.08 


.25** 


8. 


Senior Rating Total 








1.00 


.26** 


.17 


-.05 


.42** 


10. 


Advisor Instructor 










1.00 


.56** 


-.02 


.28** 


15. 


CEQ Instructor 












1.00 


.00 


.27* 


18. 


Publications 














1.00 


.07 


3. 


Colleague Rating 
















1.00 



*p < .05 
**p < .01 



a multiple correlation and the weights for each prediction were computed. The 
multiple correlation was found to be .52 and the standardized and unstandardized 
regression equations are presented below: 



2y « (.22?5) Academic Rank + (.0768) Advisor + (.1727) CEQ + (.OlSA) 

Publications (.3849) Senior Rating 
Y - -7.3658 + (.5998) Academic Rank + (.6183) Advisor + (1.5949) CEQ 

+ (.0077) P^jblications + (.0325) Senior Rating 
Where 

" predicted Colleague Rating (standardized) 
Y ■• predicted Colleague Rating (unstandardized) 
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Discussion 

The results presented indicate that even with the addition of senior 
ratings, the academic rank of an instructor is more highly related to publi- 
cations than any of the other variables* Senior Rating also failed to show 
a significant relationship with publications. Senior Rating for the Total, 
1969 and 1972 showed significant relationships to Colleague and Advisor 
Instructor ratings and high but not significant relationships to the CEQ 
Instructor variable. 

The high relationship between colleague and senior ratings indicates 
that they are more similar in their judgments of good teachers than the 
currently enrolled students and colleagues are. This seems to imply that 
either; (a) graduating seniors have somewhat modified their attitudes regarding 
who the excellent teachers are so that they are more closely related to 
colleague perceptions, or (b) that graduating seniors and faculty have had 
more time to share opinions which may not be as readily available to currently 
enrolled students. It Is interesting to note, however, that the senior ratings 
were also highly and significantly related to current student ratings. 
Another interesting result is the fact that the 1969 Senior Rating were 
more highly related to the Colleague rating than to the CEQ Instructor rating 
but the 1972 Senior Rating reversed that situation. This suggests that 
student ratings may be more stable over time than McKeachle expected especially 
if we realize that some of the current students who were rating in 1969 would 
be graduating seniors in 1972. This particular portion of the study seems to 
support the claims made earlier by Drucker and Remmers. 

In spite of the high relationship between colleague and senior ratings, 
it still appears that students and teachers differ in the basis of their 
rating since colleagues appear to take into consideration the academic rank 
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of the iustructor. The hypothesis in the earlier study by Aleamoni and Yimer 
that such a relationship is explainable in terms of reputation and that 
instructors who are at a university longer are more apt to be known to more 
colleagues, still appears tenable for colleague ratings but not for graduating 
senior ratings. It seems that graduating seniors are still relying on what 
they observed in the classroom in arriving at judgments of excellence in 
teaching* 

The present study supports the conclusion reached by Gaff that senior 
ratings and colleague ratings are related* However, since no correlational 
data was provided by Gaff, we have no way of determining if the magnitude of 
our relationships is comparable. 

The multiple correlation of .52 represents a significant increase over 
the multiple correlation of .40 found In the previous study and is largely due 
to the addition of the Senior Rating. As expected, the Senior Rating is the 
largest contributor to predicting Colleague Rating with Academic Rank and CEQ 
Instructor also contributing significantly. 

This study has added a meaningful dimension to the earlier stu' ' by 
Aleamoni and Yimer by incorporating graduating senior rating data, however, 
the usefulness of such ratings needs to be more thoroughly explored. Issues 
such as ; (a) why colleague and senior ratings are highly related but yet not 
similarly related to academic rank, (b) why ratings by seniors in the same 
year differ when compared to currently enrolled student ratings in earlier 
years, and (c) how stable are specific student ratings when they become 
seniors, also need to be investigated much more thoroughly. 
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